Search for: All records

Background. Simulation has revolutionized teaching and learning. However, traditional manikins are limited in their ability to exhibit emotions, movements, and interactive eye gaze. As a result, students struggle with immersion and may be unable to authentically relate to the patient. Intervention. We developed a new type of patient simulator called the Physical-Virtual Patients (PVP) which combines the physicality of manikins with the richness of dynamic visuals. The PVP uses spatial Augmented Reality to rear project dynamic imagery (e.g., facial expressions, ptosis, pupil reactions) on a semi-transparent physical shell. The shell occupies space and matches the dimensions of a human head. Methods. We compared two groups of third semester nursing students (N=59) from a baccalaureate program using a between-participant design, one group interacting with a traditional high-fidelity manikin versus a more realistic PVP head. The learners had to perform a neurological assessment. We measured authenticity, urgency, and learning. Results. Learners had a more realistic encounter with the PVP patient (p=0.046), they were more engaged with the PVP condition compared to the manikin in terms of authenticity of encounter and cognitive strategies. The PVP provoked a higher sense of urgency (p=0.002). There was increased learning for the PVP group compared to the manikin group on the pre and post-simulation scores (p=0.027). Conclusion. The realism of the visuals in the PVP increases authenticity and engagement which results in a greater sense of urgency and overall learning. 

Virtual content into a real environment. There are many factors that can affect the perceived physicality and co-presence of virtual entities, including the hardware capabilities, the fidelity of the virtual behaviors, and sensory feedback associated with the interactions. In this paper, we present a study investigating participants’ perceptions and behaviors during a time-limited search task in close proximity with virtual entities in AR. In particular, we analyze the effects of (i) visual conflicts in the periphery of an optical see-through head-mounted display, a Microsoft HoloLens, (ii) overall lighting in the physical environment, and (iii) multimodal feedback based on vibrotactile transducers mounted on a physical platform. Our results show significant benefits of vibrotactile feedback and reduced peripheral lighting for spatial and social presence, and engagement. We discuss implications of these effects for AR applications. 

Embodied virtual agents serving as patient simulators are widely used in medical training scenarios, ranging from physical patients to virtual patients presented via virtual and augmented reality technologies. Physical-virtual patients are a hybrid solution that combines the benefits of dynamic visuals integrated into a human-shaped physical form that can also present other cues, such as pulse, breathing sounds, and temperature. Sometimes in simulation the visuals and shape do not match. We carried out a human-participant study employing graduate nursing students in pediatric patient simulations comprising conditions associated with matching/non-matching of the visuals and shape. 

Conventional Intelligent Virtual Agents (IVAs) focus primarily on the visual and auditory channels for both the agent and the interacting human: the agent displays a visual appearance and speech as output, while processing the human’s verbal and non-verbal behavior as input. However, some interactions, particularly those between a patient and healthcare provider, inherently include tactile components.We introduce an Intelligent Physical-Virtual Agent (IPVA) head that occupies an appropriate physical volume; can be touched; and via human-in-the-loop control can change appearance, listen, speak, and react physiologically in response to human behavior. Compared to a traditional IVA, it provides a physical affordance, allowing for more realistic and compelling human-agent interactions. In a user study focusing on neurological assessment of a simulated patient showing stroke symptoms, we compared the IPVA head with a high-fidelity touch-aware mannequin that has a static appearance. Various measures of the human subjects indicated greater attention, affinity for, and presence with the IPVA patient, all factors that can improve healthcare training. 

The field of intelligent virtual agents (IVAs) has evolved immensely over the past 15 years, introducing new application opportunities in areas such as training, health care, and virtual assistants. In this survey paper, we provide a systematic review of the most influential user studies published in the IVA conference from 2001 to 2015 focusing on IVA development, human perception, and interactions. A total of 247 papers with 276 user studies have been classified and reviewed based on their contributions and impact. We identify the different areas of research and provide a summary of the papers with the highest impact. With the trends of past user studies and the current state of technology, we provide insights into future trends and research challenges.